Positional recognition for augmented reality environment

ABSTRACT

A method includes: receiving, in a first device, a relative description file for physical markers that are positioned at locations, the relative description file defining relative positions for each of the physical markers with regard to at least another one of the physical markers; initially localizing a position of the first device among the physical markers by visually capturing any first physical marker of the physical markers using an image sensor of the first device; and recognizing a second physical marker of the physical markers and a location of the second physical marker without a line of sight, the second physical marker recognized using the relative description file.

TECHNICAL FIELD

This document relates, generally, to positional recognition for anaugmented reality environment.

BACKGROUND

In the context of computer-based consumption of media and other content,it is becoming increasingly common to provide the participant withimmersive experiences. One field involves the presentation of virtualreality (VR) and/or augmented reality (AR) environments on a device suchas a smartphone or a tablet. In an AR environment, a person can watch ascreen that presents at least both an aspect of a physical environment(e.g., a video or image of a physical space) and an aspect of VR (e.g.,a virtual object superimposed on the video or image).

SUMMARY

In a first aspect, a method includes: receiving, in a first device, arelative description file for physical markers that are positioned atlocations, the relative description file defining relative positions foreach of the physical markers with regard to at least another one of thephysical markers; initially localizing a position of the first deviceamong the physical markers by visually capturing any first physicalmarker of the physical markers using an image sensor of the firstdevice; and recognizing a second physical marker of the physical markersand a location of the second physical marker without a line of sight,the second physical marker recognized using the relative descriptionfile.

Implementations can include any or all of the following features. Atleast the first physical marker can include a code. At least the firstphysical marker can include a natural marker. The method can furtherinclude presenting an augmented reality environment on a display of thefirst device, the augmented reality environment including at least animage captured using a camera of the first device and at least a firstvirtual object applied to the image. The first virtual object can bepositioned in the image at a predefined location relative to the firstphysical marker. The position of the first device can initially belocalized by aiming the camera in a first direction toward the firstphysical marker, and after the position of the first device is initiallylocalized, a user can aim the camera in a second direction, andrecognizing the second physical marker can include determining, usingthe relative description file, that a location of the second physicalmarker is located in the second direction. The method can furtherinclude updating the augmented reality environment with a second virtualobject in a predefined position relative to the second physical marker.The user can move the first device to the location of the secondphysical marker, and at the location of the second physical marker thefirst physical marker may no longer be visible in the augmented realityenvironment and the user can aim the camera in a second direction towardthe location of the first physical marker without a line of sight, andthe method can further include updating the augmented realityenvironment with the first virtual object at the predefined locationrelative to the first physical marker. Before the second physical markeris recognized, the second physical marker can be removed from thelocation of the second physical marker, and the second physical markerand the location of the second physical marker can be recognized usingthe relative description file. Initially localizing the position of thefirst device can include centering a coordinate system of the firstdevice based on the location of the first physical marker, whereinrecognizing the second physical marker does not comprise re-centeringthe coordinate system. Centering the coordinate system of the firstdevice can include recalibrating a center of the coordinate system tocorrect a tracking error. The method can further include providing therelative description file to a second device, wherein the second deviceis configured to initially localize a position of the second deviceamong the physical markers by visually capturing any one of the physicalmarkers using an image sensor of the second device, and to use therelative description file to recognize locations of all remaining onesof the physical markers without a line of sight. The relativedescription file can be a first relative description file, and themethod can further include receiving a second relative description filefrom a second device, and at least partially merging the first andsecond relative description files. The relative description file candefine the relative position of the second physical marker with regardto the first physical marker, the relative position comprising adisplacement of the second physical marker relative to the firstphysical marker. The first device can recognize at least some of thephysical markers, and the method can further include generating a log onthe first device reflecting the at least some of the physical markersthat the first device has recognized. The relative description file candefine the relative position of each of the physical markers with regardto all other ones of the physical markers.

In a second aspect, a non-transitory storage medium can have storedthereon instructions that when executed are configured to cause aprocessor to perform operations, the operations comprising: receiving,in a first device, a relative description file for physical markers thatare positioned at locations, the relative description file definingrelative positions for each of the physical markers with regard to atleast another one of the physical markers; initially localizing aposition of the first device among the physical markers by visuallycapturing any first physical marker of the physical markers using animage sensor of the first device; and recognizing a second physicalmarker of the physical markers and a location of the second physicalmarker without a line of sight, the second physical marker recognizedusing the relative description file.

In a third aspect, a system includes: a first device configured toreceive, for each of physical markers that are positioned at locations,location points determined using the first device, each of the locationpoints associated with a respective one of the physical markers, thefirst device further configured to generate a relative description filefor the physical markers based on the location points, the relativedescription file defining relative positions for each of the physicalmarkers with regard to at least another one of the physical markers; anda second device and a third device both configured to receive therelative description file generated by the first device, wherein each ofthe second and third devices is further configured to initially localizea position of the second or third device, respectively, among thephysical markers by visually capturing any one of the physical markersusing an image sensor of the second or third device, respectively, andto use the relative description file to recognize locations of allremaining ones of the physical markers without a line of sight.

In a fourth aspect, a method includes: receiving, in a first device andfor each of physical markers that are positioned at locations, locationpoints determined using the first device, each of the location pointsassociated with a respective one of the physical markers; generating arelative description file for the physical markers based on the locationpoints, the relative description file including relative positions foreach of the physical markers with regard to at least another one of thephysical markers; and providing the relative description file to atleast a second device and a third device, wherein each of the second andthird devices is configured to initially localize a position of thesecond or third device, respectively, among the physical markers byvisually capturing any one of the physical markers using an image sensorof the second or third device, respectively, and to use the relativedescription file to recognize locations of all remaining ones of thephysical markers without a line of sight.

Implementations can include any or all of the following features. Atleast one of the physical markers can include a code. At least one ofthe physical markers can include a natural marker.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 shows an example of a system that can be used for generating animmersive experience in an augmented reality (AR) environment.

FIGS. 2A-D show examples of devices with AR environments.

FIG. 3 shows an example of devices sharing relative description filesand merging them with each other.

FIG. 4 shows an example of a method involving generation of a relativedescription file (RDF).

FIG. 5 shows an example of a method involving recognition of markersusing an RDF.

FIG. 6 shows an example of a method that involves at least one devicesharing an RDF and merging the RDF with a received RDF.

FIG. 7 shows an example of a computer device and a mobile computerdevice consistent with disclosed embodiments.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

This document describes examples of recognizing markers and theirrelative positions and orientation in an augmented reality (AR)environment. The AR environment can include imagery of a physicalreality (e.g., a camera view of the user's surroundings) and imagery ofvirtual reality (e.g., an AR object). The presentation of the ARenvironment can then provide the user a view that simultaneously showsat least some of the imagery of the physical reality and at least someof the imagery of virtual reality. In some implementations, a relativedescription file (RDF) can define relative positions between aparticular set of physical markers that have been placed at respectivelocations. Such an RDF can be comparatively limited in size and easy toshare with multiple devices. A device having the RDF can localize itsown position by recognizing one of the physical markers (e.g., using acamera) to extrapolate the device position and orientation, and can usethe RDF to learn, and act based on, the locations and identities of theother physical markers, also when these markers are not currentlyvisible or directly detectable by the device. Some implementations canidentify both the relative positions and orientations of physicalmarkers, whereas certain examples herein mention only identification ofpositions for brevity. Some implementations can also or instead beapplied in a virtual-reality system, for example to provide an immersivevirtual-reality experience.

Existing systems that handle AR environments have attempted to usedistinctive and recognizable items (such as a QR code) in the physicalenvironment as anchor points for virtual objects. However, such systemscan be dependent on the availability of a line-of-sight to the item. Asanother example, after a first such marker is recognized, therecognition of a second (or subsequent) marker can trigger are-centering of the applied coordinate system. When the device no longersees the marker, it can lose the context of position because such systemmay not extrapolate the position within the world after it loses sightof the marker. That is, the systems are typically configured to treatthe captured marker as the device's “center-of-world” for purposes ofpresenting an AR environment, such that the AR coordinate system is thencentered on the location where the marker was situated. One drawback ofthis approach is that if more than one marker is used, the device willin a sense re-center its coordinate system whenever it captures the nextmarker. Another drawback is that when multiple devices areparticipating, it can become challenging to have a common understandingas to where the center-of-world is.

Some previous systems have attempted to use an area description file(ADF) to facilitate the use of markers in an AR context. Creating theADF could be a cumbersome process because it may be necessary to walkaround a room with a device and have the camera pick up details of allthe structures and objects that are present. This approach is based oncreating, at a preliminary stage, a comprehensive understanding of thephysical location. The resulting ADF can be relatively large, dependingon the size of the physical space and the complexity of the structurestherein. In a multi-device scenario, where several devices should beusing the same ADF in order to facilitate a common AR experience for theusers, distributing the large ADF to all devices can take time and canplace significant demands on communication networks.

Recently, some AR systems have taken a somewhat different approach todevice orientation. For example, some systems are configured to performenvironmental tracking using multiple lenses on a handheld device suchas a smartphone or tablet. The system can use two lenses and infraredsignals to effectively triangulate its position within the room. Thiscan allow the Tango system to have stereo vision and develop astereoscopic understanding of the room, including its depth, and whatsurfaces and objects are in the room. The system can conveniently mapthese features to facilitate its spatial understanding. Other systemsoffer AR functionality using comparable techniques.

One characteristic of some AR systems is an ability to extrapolate fromwhere the sensing device is in the room. The system can rely on alocalization point in the physical world to perform continuousextrapolation of position, to extend its understanding of the structureof the space from that localization point. One benefit of this is thatthe system is not dependent on having a line-of-sight to a known marker.Rather, by extrapolating from the current position of the device, thesystem can deduce where the marker is and can perform an AR functionbased on it, such as to place a virtual object in the locationcorresponding to the marker.

In some implementations, a system can use a physical marker for at leasttwo beneficial purposes: to indicate a position in space, for example todefine a center-of-world for the device, and to use one or more markersas real-world cues to an audience of where they can expect AR objects toappear. For example, an organizer of an educational session such as alecture or a tour can place markers throughout a room or other physicallocation. The organizer can capture the individual markers using ahandheld device, so as to define for the AR system which marker islocated where. When attendees use their own devices to capture themarkers, selected virtual objects can become visible in the ARenvironments that are presented on their respective devices. Also, basedon knowing the relative locations of the markers with respect to eachother, the AR system can show the attendee a virtual object,corresponding to a marker, before the attendee's device directly detectsthat marker. Examples are described below.

FIG. 1 shows an example of a system 100 that can be used for generatingan immersive experience by way of an AR environment. In someimplementations, the immersive experience allows the user to see one ormore AR objects in combination with imagery of physical reality. Forexample, the immersive experience can be organized in form of a tourthat includes multiple AR objects positioned at respective locationsrelative to the physical reality. The system 100 is here shown in aspace 102, which can be any kind of physical space, including, but notlimited to, a room or other indoor or outdoor premises. The space 102 ishere schematically shown from above. Tables 104A-B in the space 102 arevisible. For example, an organizer of the tour may be planning that theparticipants should visit several positions at the tables 104A-B andthere be presented with AR experiences as part of the tour. Any otherphysical structures than tables can be used instead or additionally.

The organizer can place multiple markers in the space 102. Here, markers106A and 106B have been placed atop the table 104A, and markers 106C and106D have been placed atop the table 104B. For example, the organizermay have decided that the markers 106A-D should be visited inalphabetical order, from 106A to 106D, or that they can be visited in arandom order. The markers 106A-D can be used to trigger the appearanceof, and define the location for, one or more virtual objects that willbe visible to tour participants, for example as will be described below.

A participant who is positioned at one of the markers 106A-D can have aline-of-sight to one or more other marker. The line-of-sight, whenavailable, can allow the participant and/or the participant's device toview (e.g., directly view along a line) the other marker from theircurrent position. However, in some situations a line-of-sight is notavailable. For example, a structure such as wall 108 may be positionedbetween two or more markers. As another example, the participant and theparticipant's device can be observing the other marker from a particularviewing point (perspective) that the other marker is currently difficultor impossible to perceive from the current position.

The organizer can use one or more devices to create the AR experience.Here, a device 110 is schematically shown in the space 102. Any kind ofdevice can be used, including, but not limited to, a smartphone or atablet device. The device 110 here includes an AR system 112. The ARsystem uses light sensing (e.g., infrared light) and two or more lensesto generate a stereo vision in the space 102 in order to develop astereoscopic understanding thereof that allows the AR system 112 to alsoextrapolate into positions of the space 102 to which there is currentlyno line-of-sight. For example, the AR system 112 can be, or can be partof, an AR system that uses a sensing device and is configured toextrapolate from where the sensing device is in the room using alocalization point in the physical world. Such AR system can performcontinuous extrapolation of position to extend its recognition of thestructure of the space from that localization point. In someimplementations, the AR system 112 is executed partially on the device110 and partially on one or more other devices (e.g., another handhelddevice or a cloud system).

The device 110 can have one or more display screens 114. In someimplementations, the display screen can be a touchscreen. For example,the display screen 114 can present a control 116 that allows theorganizer to create a tour using the device 110 and (here) the markers106A-D. Having activated the AR system 112 using the control 116, theorganizer can visit the distributed markers 106A in any order. Here, forexample, the organizer visits locations 118A through 118D in that order,to capture the respective markers 106A-D. At each location, the devicecan capture the corresponding marker and make a record of its location.For example, the organizer can trigger the capture using a control 120that can be presented on the display screen 114. As another example, thedevice 110 can continuously monitor the signal(s) from its sensor(s) forthe presence of markers such as the markers 106A-D and can capture suchmarker(s) whenever detected. When the device captures any of the markers106A-D, an identity of the marker and a position of the marker can beregistered by the AR system 112. For example, such registration can herereflect that the marker 106A is positioned at the location 118A, thatthe marker 106B is positioned at the location 118B, that the marker 106Cis positioned at the location 118C, and that the marker 106D ispositioned at the location 118D. The position of any or all markers canbe defined relative to a known position, such as to the position of ananchor marker or other localization point, for example as will bedescribed below.

The initial localization of the device 110 (against any of the markers106A-D in this example) may not necessarily imply that this particularmarker is then defined as the (0,0,0) of the AR world. Rather, thelocalization marker can be defined by way of an offset relative to anRDF-defined center of the world. Such a center may have been establishedby an earlier localization (e.g., by the person who arranged/defined themarkers in the physical environment), or by a definition (such as bychoosing the North Pole).

The AR system 112 can generate one or more data structures based on thecapturing of the markers 106A-D done with the device 110. Here, arelative description file (RDF) 122 is generated. The RDF 122 definesrelative positions for each of the markers 106A-D with regard to atleast another one of the markers 106A-D. In some implementations, themarker 106A can be considered as an anchor marker. The RDF 122 candefine the relative position of the marker 106B based on the detectedposition of the marker 106A; the relative position of the marker 106Ccan be defined based on the detected position of the marker 106B; andthe relative position of the marker 106D can be defined based on thedetected position of the marker 106C. For example, if the location ofthe marker 106A is defined to be (0,0,0) using a Cartesian coordinatesystem for the space 102, then the relative position of the marker 106Bcan be defined as the displacement (50,100,0) relative to the positionof the marker 106A. In some implementations, the RDF 122 defines therelative location of each of the markers 106A-D with regard to all theother ones of the markers 106A-D.

The markers 106A-D are recognizable by the sensor(s) on the device 110so that they can be captured accordingly. One or more suitable forms ofmarker can be used. In some implementations, a code can be used. A codecan be a digital code. For example, the marker can include a QR codethat is detected by a light sensor (e.g., a camera) on the device 110.In some implementations, a non-code marker can be used. For example, anatural marker can be any relatively flat image that has a sufficientnumber of features, is rich in contrast, and/or has unique shapes. Thenatural marker can be detected by a light sensor (e.g., a camera) on thedevice 110. In some implementations, a marker can comprise one or morefeatures occurring in the physical environment. Such a marker can bedefined without physically intervening in the physical environment. Anobject situated in the physical environment can be defined as a marker.For example, a natural marker can be used, including, but not limitedto, a unique photograph with high contrast edges, or a geometric designwith unique edges. A real-world marker can be used, including, but notlimited to, a photo-captured image of, say, the grain on a table, whichcan be used analogous to a fingerprint. With at least natural markersand real-world markers, the system looks for a set of image-basedfeatures. Such marker codes can be more literally translated into bitsand bytes, in some implementations. Based on the captured image(s), anassociated marker identity can be determined.

As such, the system 100 can be used to perform one or more processesthat are relevant to marker definition and/or recognition. For example,the device 110 can receive location points for each of the markers106A-D corresponding to the locations 118A-D. The location points can beused for generating the RDF 122. The RDF 122 can define relativepositions for each of the markers 106A-D with regard to at least anotherone of the markers 106A-D. Such an RDF 122 can be provided to one ormore devices for use in providing an AR experience, for example as willbe described below.

In some implementations, multiple AR entities can be represented inseparate groupings organized by respective physical markers. Forexample, users who have different interest can explore such groupings indifferent order from each other. In some implementations, a teacherusing an implementation can split users into groups among tables in aclassroom for space management. The teacher can move the groups betweenthe tables to inspect new AR objects or AR environments. For example,each of the groups of users can remain in a respective area of theclassroom, and can then take turns teaching the rest of the users aboutthe AR object(s) that their group has explored. The users in othergroups, positioned elsewhere in the classroom, can aim their devicestoward the presenting group and see the relevant AR object(s).

FIGS. 2A-D show examples of devices with AR environments. In FIG. 2A, adevice 200 is shown, such as a handheld device with at least one displayscreen 202. The device 200 can be similar or identical to the device 110(FIG. 1). An AR environment 204 can be presented on the display screen202. The AR environment 204 can include at least one image 206 capturedusing a camera of the device 200. For example, a floor 208 and part of atabletop surface 210 are here visible in the image 206. In someimplementations, the tabletop surface 210 can correspond to the table104A (FIG. 1) on which the markers 106A-B were placed. For example, theAR environment 204 can here reflect that the marker 106A has beencaptured. The marker 106A is a physical object detectable by a cameraand is here visible in the image 206 of the AR environment 204. Theimage 206 can be a still image, or can be a continuous feed ofessentially live video captured by the camera of the device 200. The ARenvironment 204 can include at least one virtual object 212 applied tothe image 206. Here, the virtual object 212 is an apple. For example,the apple is the AR element that has been assigned to the marker 106A.The virtual object 212 is placed at a position in a predefined locationrelative to the marker 106A. The virtual object 212 has been applied tothe tabletop surface 210 in this example, but could instead oradditionally be applied to any other part or aspect of the image 206,such as by having the AR element move within the AR environment 204.

The device 200 includes an AR system 214 as schematically indicated. Forexample, the AR system can be identical or similar to the AR system 112(FIG. 1). The AR system 214 can receive, maintain and use one or moreRDF 216. The RDF 216 can be similar or identical to the RDF 122 (FIG.1). In some implementations, a device (e.g., the device 110 in FIG. 1)can provide the RDF 216 to one or more devices, such as the device 200.The device 200 can be configured to initially localize its positionamong multiple markers (e.g., the markers 106A-D in FIG. 1) by visuallycapturing an arbitrary one of the markers. For example, the device 200can include an image sensor (e.g., a camera) and can visually captureany or all of the markers using the image sensor. The device 200 canalso be configured to use the RDF 216 to recognize all remaining ones ofthe markers, and their locations, without a line of sight. For example,after the device 200 localizes at one of the markers 106A-D (FIG. 1),the RDF 216 will inform the AR system 214 about the identity andlocation of all others of the markers 106A-D, also when the device 200does not have a direct line-of-sight to any such marker(s).

The AR system 214 can use one or more sensors 218 of the device 200 tocapture markers. In some implementations, a light sensor can be used.For example, a sensor for infrared light can use two lenses on thedevice 200 to obtain a stereoscopic view of the physical location.

The signals from the sensor(s) 218 can be interpreted by one or moreimage analyzers 220. For example, the image analyzer 220 can read avisual symbol that includes a QR code and inform the AR system 214 aboutthe information encoded in the symbol. As another example, the imageanalyzer 220 can capture an image of a natural marker, and compare atleast part of the captured image with a mapping of images toidentifiers, to determine the identity of the captured natural marker.

The RDF 216 can allow the user to experience a variety of aspects aspart of the tour defined by the distributed markers. For example, afterthe user localizes the device 200, such as by capturing the marker 106Aas indicated in FIG. 2A, the user can aim the sensor 218 (e.g., a cameraor other light sensor) in some direction, and the system will indicatewhether any markers are positioned along that direction. Assume, forexample, and with reference again to FIG. 1, that the user is standingat the location 118A and aims the sensor of the device 200 toward thelocation 118C where the marker 106C is positioned. The line-of-sight maythen be interrupted by the wall 108. That is, the device 200 does nothave a line-of-sight from the location 118A toward the location 118C.FIG. 2B shows an example of what the display screen 202 of the device200 can present in such a situation. In the AR environment 204, anupdated image 206′ shows that the wall 108 is substantially blocking theline-of-sight in the current direction. The table 104B is obscured bythe wall 108 but is indicated in phantom for clarity.

The AR system 214 of the device 200 can access the RDF 216 which definesthe relative positions of the available markers. Here, the device iscurrently positioned at the location 118A and is aimed in the directionof the table 104B. The RDF 216 can then indicate, based on the currentlocation of the device 200 and on the current direction of the sensor,that the marker 106C is positioned in this direction. That is, the ARsystem 214 can extrapolate beyond the wall 108 and can determine thatbut for the wall 108 the marker 106C would be within the currentline-of-sight. The AR system 214 can therefore recognize the marker 106Cin the current situation although the marker 106C is not currentlyvisible to the device. Based on the recognition of the marker 106C, theAR system 214 can place one or more virtual objects 222 in the ARenvironment 204. For example, the virtual object 222 is here a bananathat is added to the image 206′ in the virtual environment 204. That is,after the position of the device 200 is initially localized using themarker 106A at the location 118A, the user can aim the camera of thedevice 200 in an arbitrary direction. Another marker (e.g., the marker106C) can then be recognized by determining, using the RDF 216, that therelative location of the marker 106C with regard to the marker 106A islocated in the current direction.

In some implementations, the AR system 214 can recognize the marker 106Cand its position in the situation described above, regardless whetherthe marker 106C is still in that location. Namely, the recognition canbe based on the RDF 216, which defines the location 118C relative to thelocation 118A, where the device 200 is currently positioned in thisexample. As such, the AR system can, based on the information in the RDF216, recognize the location 118C as being the location where the marker106C was originally captured, and can present the virtual object 222 inthe AR environment 204 based on this recognition. That is, the marker106C can be removed from the location 118C and can thereafternevertheless be recognized by the AR system 214 based on the RDF 216.

The AR system 214 can log one or more events regarding the device 200.In some implementations, the AR system 214 can generate a log on thedevice 200 reflecting the markers 106A and 106C that the device 200 hasrecognized in this example. If the device had also recognized othermarkers (e.g., one or more of the markers 106B and 106D in FIG. 1) thiscan be reflected in the log.

The RDF 216 can facilitate recognition of any marker whose relativeposition is defined in the RDF 216. Assume, for example, that the usermoves to the location 118C (FIG. 1) and aims the camera of the device200 in the direction toward the location 118A. The RDF 216 can thenindicate, based on the current location of the device 200 and on thecurrent direction of the sensor, that the marker 106A is positioned inthis direction. FIG. 2C shows that the virtual object 212 has been addedto an image 206″ of the AR environment 204 presented on the displayscreen 202. That is, although the table 104A (here schematicallyindicated) is obscured by the wall 108, the AR system 214 cannevertheless extrapolate beyond this interruption of the line-of-sight,recognize the non-visible marker 106A, and place the virtual object 212in the correct position. For example, if the device 200 for any reasonhad lost its localization by the time the device 200 was positioned atthe location 118C, the user can relocalize the device 200 against themarker 106C and thereafter aim the camera elsewhere, such as toward thelocation 118A. Because the device 200 has again been localized in thisexample, and because the RDF 216 indicates the positions of virtualobjects relative to the device 200, the system can place the virtualobject 212 so as to be visible to the user from the current location,although the marker 106A (FIG. 2A) itself is not currently detected bythe device 200.

The initial localization of the device, such as at the marker 106A inthe location 118A, can correspond to a centering of a coordinate systemof the device 200 on that location. For example, the AR system 214 candefine the coordinate system based on readings from the one or moresensors 218. Moreover, when the AR system 214 then detects anothermarker, such as the marker 106D if the device 200 is moved to thelocation 118D, the AR system 214 may not re-center the coordinatesystem. Rather, the AR system 214 can recognize the marker 106D based onthe relative positions defined in the RDF 216, and can place a virtualobject in a position based on the recognized marker, withoutre-centering its coordinate system in doing so. Although the coordinatesystem is not re-centered, a center of the coordinate system can bere-calibrated. For example, this can be done to correct a trackingerror. In some implementations, a subsequent marker (e.g., the marker106C) can be used to relocalize the device 200 such that any trackinginaccuracies in relocating the device 200 from, say, the marker 106A tothe marker 106C are rectified.

In some implementations, the position of the device 200 can initially belocalized by centering a coordinate system of the device based on thelocation of a physical marker. Another physical marker can then be usedfor re-centering the coordinate system. In so doing, any virtual objects(e.g., the virtual objects 212, 222) can continue to be presented in thesame space. In some implementations, this can occur by way of camerapassthrough, for example because the relative coordinates areproportionally the same. As such, re-centering the system but keepingvirtual objects in the same place is possible. That is, the wholevirtual world can shift, but the relative positions of the markers canstay the same.

The RDF 216 can be shared with one or more devices. For example, FIG. 2Dshows a device 224 that on its display screen 226 presents an ARenvironment 228 that currently includes an image 230 having a virtualobject 232 applied to it. The device 224 can be similar or identical tothe device 200. For example, the device 224 visits the location 118B(FIG. 1) and there recognizes the marker 106B, wherein the virtualobject 232 is triggered by the recognition of that marker. The device224 can initially localize its position in the space 102 (FIG. 1) bycapturing any of the markers 106A-D. For example, the presentation ofthe virtual object 232 can be triggered by the localization of thedevice 224 at the location 118B, or the device 224 may have previouslybeen localized at another one of the markers, and can thereafterrecognize the marker 106B based on the RDF 216 that it has received.That is, two devices having the same RDF (e.g., the RDF 216) canlocalize their position at the same or different markers, and then havea common spatial understanding of the space where the markers aredistributed. The RDF 216 can be shared with the device 224 by anotherparticipant device (e.g., the device 200) or by the device of anorganizer (e.g., the device 110 in FIG. 1) to name just two examples.

FIG. 3 shows an example of devices sharing RDFs and merging them witheach other. Here, physical markers 300A-C have been distributed in aphysical space. An RDF 302A is here used by (e.g., stored on) a firstdevice, and an RDF 302B is used by (e.g., stored on) a second device.The first device in this example visits only the markers 300A and 300B,but not the marker 300C. For example, the first device localizes itsposition at the marker 300A and thereafter captures the marker 300B, orvice versa. The RDF 302A therefore defines a relative position 304A thatconnects the markers 300A-B. For example, the relative position 304A cancomprise a displacement of the marker 300B relative to the marker 300A.In contrast, the RDF 302A does not define any relative positionregarding the marker 300C which has not been captured by the firstdevice. In fact, the marker 300C may not be present in the RDF 302A insuch situations but is shown here for purposes of explanation.

The second device in this example, moreover, visits only the markers300B and 300C, but not the marker 300A. For example, the second devicelocalizes its position at the marker 300B and thereafter captures themarker 300C, or vice versa. The RDF 302B therefore defines a relativeposition 304B that connects the markers 300B-C. For example, therelative position 304B can comprise a displacement of the marker 300Crelative to the marker 300B. In contrast, the RDF 302B does not defineany relative position regarding the marker 300A which has not beencaptured by the second device. In fact, the marker 300A may not bepresent in the RDF 302B but is shown here for purposes of explanation.Either or both of the first and second devices can share itscorresponding RDF 302A or 302B with another device, such as with theother one of the first and second devices.

A device that receives an RDF can at least in part merge the receivedRDF with one or more RDFs already known to the device. Here, anoperation 306 schematically illustrates that the RDFs 302A-B are beingmerged with each other to form an RDF 302C. The RDF 302C reflectsrelative positions regarding all of the markers 300A-C. That is, the RDF302C includes both the relative positions 304A and 304B. From theperspective of the first device, which initially had identified therelative locations only of the markers 300A-B, the RDF 302C correspondsto the increased knowledge of also knowing the relative position of themarker 300C. Similarly, from the perspective of the second device, whichinitially had identified the relative locations only of the markers300B-C, the RDF 302C corresponds to the increased knowledge of alsoknowing the relative position of the marker 300A. That is, the firstdevice was able to learn about the marker 300C by way of the RDF 302Bshared from the second device which had captured that marker, becausethe second device had also captured the marker 300B which was alreadyknown to the first device.

FIGS. 4-6 show examples of methods. Any or all operations of a methodcan be performed using one or more processors executing instructionsstored in a non-transitory storage medium, for example as describedbelow. In some implementations, more or fewer operations than shown canbe performed. In some implementations, two or more operations can beperformed in a different order. Some components introduced elsewhere inthis disclosure will be mentioned in the following description, solelyfor purposes of illustration.

FIG. 4 shows an example of a method 400 involving generation of arelative description file (RDF). At 410, markers can be placed. Forexample, an organizer can place the markers 106A-D in the space 102. At420, markers can be captured. For example, the device 110 can capturethe markers 106A-D at the respective locations 118A-D. At 430, an RDFcan be generated. For example, the RDF 122 can be generated for themarkers 106A-D. At 440, the RDF can be provided to one or more devices.For example, the device 110 can share the RDF 122 with the device 200and with the device 224. The device 200 and/or 224 can then use theprovided RDF to localize its position and recognize one or more markers.

At 450, the marker placement can be updated. For example, the organizercan change the position of one or more of the markers 106A-D, or can addone or more new physical markers. At 460, the updated marker(s) can becaptured. For example, the organizer can capture the updated marker(s)in the space 102 using the device 110. At 470, an updated RDF can begenerated. For example, the device 110 can update the RDF 122. At 480,the updated RDF can be provided to one or more devices. For example, thedevice 110 can share the updated RDF 122 with the device 200 and withthe device 224. The device 200 and/or 224 can then use the providedupdated RDF to localize its position and recognize one or more markers.

FIG. 5 shows an example of a method 500 involving recognition of markersusing an RDF. At 510, an RDF can be received. For example, the device200 can receive the RDF 216. At 520, localization against a physicalmarker can be performed. For example, the device 200 can localize itsposition by capturing any of the markers 106A-D. At 530, a virtualobject (VO) can be displayed in an AR environment. For example, thedevice 200 can present the virtual object 212 in the AR environment 204.At 540, the device can move to the next marker. For example, the device200 can be moved from the location 118A to the location 118C. At 550,the device can recognize the next marker. For example, the device canrecognize the marker 106C at the location 118C. The device 200 canrecognize the marker 106C by way of the sensor 218 on the device 200capturing an image of the marker 106C, or by way of an extrapolationusing the RDF 216 that indicates the marker 106C to the device 200. At560, the user can aim the camera of the device in a direction. Forexample, while at the location 118C the device 200 can be aimed towardthe location 118A. At 570, a marker can be recognized. For example, thedevice 200 while at the location 118C can recognize the marker 106A atthe location 118A, either because a line-of-sight to the marker 106A isavailable, or by way of the RDF 216. At 580, one or more markers can beregistered in a log. For example, the device 200 can log the markers106A and 106C.

FIG. 6 shows an example of a method 600 that involves at least onedevice sharing an RDF and merging the RDF with a received RDF. At 610,an RDF can be generated. For example, the second device mentioned withregard to FIG. 3 can generate the RDF 302B. At 620, the RDF can beshared with at least one device. For example, the second device canshare the RDF 302B with the first device mentioned with regard to FIG.3. At 630, an RDF can be received. For example, the second device canreceive the RDF 302A from the first device. At 640, two or more RDFs canbe merged at least in part. For example, the second device can merge theRDFs 302A-B to form the RDF 302C. That is, the second device can learnabout the marker 300A, by way of the RDF 302A shared from the firstdevice which had captured that marker, because the first device had alsocaptured the marker 300B which was already known to the second device.

FIG. 7 shows an example of a computer device and a mobile computerdevice that can be used to implement the techniques described here. FIG.7 shows an example of a generic computer device 700 and a generic mobilecomputer device 750, which may be used with the techniques describedhere. Computing device 700 is intended to represent various forms ofdigital computers, such as laptops, desktops, tablets, workstations,personal digital assistants, televisions, servers, blade servers,mainframes, and other appropriate computing devices. Computing device750 is intended to represent various forms of mobile devices, such aspersonal digital assistants, cellular telephones, smart phones, andother similar computing devices. The components shown here, theirconnections and relationships, and their functions, are meant to beexemplary only, and are not meant to limit implementations of theinventions described and/or claimed in this document.

Computing device 700 includes a processor 702, memory 704, a storagedevice 706, a high-speed interface 708 connecting to memory 704 andhigh-speed expansion ports 710, and a low speed interface 712 connectingto low speed bus 714 and storage device 706. The processor 702 can be asemiconductor-based processor. The memory 704 can be asemiconductor-based memory. Each of the components 702, 704, 706, 708,710, and 712, are interconnected using various busses, and may bemounted on a common motherboard or in other manners as appropriate. Theprocessor 702 can process instructions for execution within thecomputing device 700, including instructions stored in the memory 704 oron the storage device 706 to display graphical information for a GUI onan external input/output device, such as display 716 coupled to highspeed interface 708. In other implementations, multiple processorsand/or multiple buses may be used, as appropriate, along with multiplememories and types of memory. Also, multiple computing devices 700 maybe connected, with each device providing portions of the necessaryoperations (e.g., as a server bank, a group of blade servers, or amulti-processor system).

The memory 704 stores information within the computing device 700. Inone implementation, the memory 704 is a volatile memory unit or units.In another implementation, the memory 704 is a non-volatile memory unitor units. The memory 704 may also be another form of computer-readablemedium, such as a magnetic or optical disk.

The storage device 706 is capable of providing mass storage for thecomputing device 700. In one implementation, the storage device 706 maybe or contain a computer-readable medium, such as a floppy disk device,a hard disk device, an optical disk device, or a tape device, a flashmemory or other similar solid state memory device, or an array ofdevices, including devices in a storage area network or otherconfigurations. A computer program product can be tangibly embodied inan information carrier. The computer program product may also containinstructions that, when executed, perform one or more methods, such asthose described above. The information carrier is a computer- ormachine-readable medium, such as the memory 704, the storage device 706,or memory on processor 702.

The high speed controller 708 manages bandwidth-intensive operations forthe computing device 700, while the low speed controller 712 manageslower bandwidth-intensive operations. Such allocation of functions isexemplary only. In one implementation, the high-speed controller 708 iscoupled to memory 704, display 716 (e.g., through a graphics processoror accelerator), and to high-speed expansion ports 710, which may acceptvarious expansion cards (not shown). In the implementation, low-speedcontroller 712 is coupled to storage device 706 and low-speed expansionport 714. The low-speed expansion port, which may include variouscommunication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet)may be coupled to one or more input/output devices, such as a keyboard,a pointing device, a scanner, or a networking device such as a switch orrouter, e.g., through a network adapter.

The computing device 700 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as astandard server 720, or multiple times in a group of such servers. Itmay also be implemented as part of a rack server system 724. Inaddition, it may be implemented in a personal computer such as a laptopcomputer 722. Alternatively, components from computing device 700 may becombined with other components in a mobile device (not shown), such asdevice 750. Each of such devices may contain one or more of computingdevice 700, 750, and an entire system may be made up of multiplecomputing devices 700, 750 communicating with each other.

Computing device 750 includes a processor 752, memory 764, aninput/output device such as a display 754, a communication interface766, and a transceiver 768, among other components. The device 750 mayalso be provided with a storage device, such as a microdrive or otherdevice, to provide additional storage. Each of the components 750, 752,764, 754, 766, and 768, are interconnected using various buses, andseveral of the components may be mounted on a common motherboard or inother manners as appropriate.

The processor 752 can execute instructions within the computing device750, including instructions stored in the memory 764. The processor maybe implemented as a chipset of chips that include separate and multipleanalog and digital processors. The processor may provide, for example,for coordination of the other components of the device 750, such ascontrol of user interfaces, applications run by device 750, and wirelesscommunication by device 750.

Processor 752 may communicate with a user through control interface 758and display interface 756 coupled to a display 754. The display 754 maybe, for example, a TFT LCD (Thin-Film-Transistor Liquid Crystal Display)or an OLED (Organic Light Emitting Diode) display, or other appropriatedisplay technology. The display interface 756 may comprise appropriatecircuitry for driving the display 754 to present graphical and otherinformation to a user. The control interface 758 may receive commandsfrom a user and convert them for submission to the processor 752. Inaddition, an external interface 762 may be provide in communication withprocessor 752, so as to enable near area communication of device 750with other devices. External interface 762 may provide, for example, forwired communication in some implementations, or for wirelesscommunication in other implementations, and multiple interfaces may alsobe used.

The memory 764 stores information within the computing device 750. Thememory 764 can be implemented as one or more of a computer-readablemedium or media, a volatile memory unit or units, or a non-volatilememory unit or units. Expansion memory 774 may also be provided andconnected to device 750 through expansion interface 772, which mayinclude, for example, a SIMM (Single In Line Memory Module) cardinterface. Such expansion memory 774 may provide extra storage space fordevice 750, or may also store applications or other information fordevice 750. Specifically, expansion memory 774 may include instructionsto carry out or supplement the processes described above, and mayinclude secure information also. Thus, for example, expansion memory 774may be provide as a security module for device 750, and may beprogrammed with instructions that permit secure use of device 750. Inaddition, secure applications may be provided via the SIMM cards, alongwith additional information, such as placing identifying information onthe SIMM card in a non-hackable manner.

The memory may include, for example, flash memory and/or NVRAM memory,as discussed below. In one implementation, a computer program product istangibly embodied in an information carrier. The computer programproduct contains instructions that, when executed, perform one or moremethods, such as those described above. The information carrier is acomputer- or machine-readable medium, such as the memory 764, expansionmemory 774, or memory on processor 752, that may be received, forexample, over transceiver 768 or external interface 762.

Device 750 may communicate wirelessly through communication interface766, which may include digital signal processing circuitry wherenecessary. Communication interface 766 may provide for communicationsunder various modes or protocols, such as GSM voice calls, SMS, EMS, orMMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others.Such communication may occur, for example, through radio-frequencytransceiver 768. In addition, short-range communication may occur, suchas using a Bluetooth, WiFi, or other such transceiver (not shown). Inaddition, GPS (Global Positioning System) receiver module 770 mayprovide additional navigation- and location-related wireless data todevice 750, which may be used as appropriate by applications running ondevice 750.

Device 750 may also communicate audibly using audio codec 760, which mayreceive spoken information from a user and convert it to usable digitalinformation. Audio codec 760 may likewise generate audible sound for auser, such as through a speaker, e.g., in a handset of device 750. Suchsound may include sound from voice telephone calls, may include recordedsound (e.g., voice messages, music files, etc.) and may also includesound generated by applications operating on device 750.

The computing device 750 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as acellular telephone 780. It may also be implemented as part of a smartphone 782, personal digital assistant, or other similar mobile device.

A user can interact with a computing device using a tracked controller784. In some implementations, the controller 784 can track the movementof a user's body, such as of the hand, foot, head and/or torso, andgenerate input corresponding to the tracked motion. The input cancorrespond to the movement in one or more dimensions of motion, such asin three dimensions. For example, the tracked controller can be aphysical controller for a VR application, the physical controllerassociated with one or more virtual controllers in the VR application.As another example, the controller 784 can include a data glove.

Various implementations of the systems and techniques described here canbe realized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations thereof.These various implementations can include implementation in one or morecomputer programs that are executable and/or interpretable on aprogrammable system including at least one programmable processor, whichmay be special or general purpose, coupled to receive data andinstructions from, and to transmit data and instructions to, a storagesystem, at least one input device, and at least one output device.

These computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor, and can be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. As used herein, the terms “machine-readable medium”“computer-readable medium” refers to any computer program product,apparatus and/or device (e.g., magnetic discs, optical disks, memory,Programmable Logic Devices (PLDs)) used to provide machine instructionsand/or data to a programmable processor, including a machine-readablemedium that receives machine instructions as a machine-readable signal.The term “machine-readable signal” refers to any signal used to providemachine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniquesdescribed here can be implemented on a computer having a display device(e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor)for displaying information to the user and a keyboard and a pointingdevice (e.g., a mouse or a trackball) by which the user can provideinput to the computer. Other kinds of devices can be used to provide forinteraction with a user as well; for example, feedback provided to theuser can be any form of sensory feedback (e.g., visual feedback,auditory feedback, or tactile feedback); and input from the user can bereceived in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in acomputing system that includes a back end component (e.g., as a dataserver), or that includes a middleware component (e.g., an applicationserver), or that includes a front end component (e.g., a client computerhaving a graphical user interface or a Web browser through which a usercan interact with an implementation of the systems and techniquesdescribed here), or any combination of such back end, middleware, orfront end components. The components of the system can be interconnectedby any form or medium of digital data communication (e.g., acommunication network). Examples of communication networks include alocal area network (“LAN”), a wide area network (“WAN”), and theInternet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

In some implementations, the computing devices depicted in FIG. 7 caninclude sensors that interface with a virtual reality (VR headset 785).For example, one or more sensors included on a computing device 750 orother computing device depicted in FIG. 7, can provide input to VRheadset 785 or in general, provide input to a VR space. The sensors caninclude, but are not limited to, a touchscreen, accelerometers,gyroscopes, pressure sensors, biometric sensors, temperature sensors,humidity sensors, and ambient light sensors. The computing device 750can use the sensors to determine an absolute position and/or a detectedrotation of the computing device in the VR space that can then be usedas input to the VR space. For example, the computing device 750 may beincorporated into the VR space as a virtual object, such as acontroller, a laser pointer, a keyboard, a weapon, etc. Positioning ofthe computing device/virtual object by the user when incorporated intothe VR space can allow the user to position the computing device to viewthe virtual object in certain manners in the VR space. For example, ifthe virtual object represents a laser pointer, the user can manipulatethe computing device as if it were an actual laser pointer. The user canmove the computing device left and right, up and down, in a circle,etc., and use the device in a similar fashion to using a laser pointer.

In some implementations, one or more input devices included on, orconnect to, the computing device 750 can be used as input to the VRspace. The input devices can include, but are not limited to, atouchscreen, a keyboard, one or more buttons, a trackpad, a touchpad, apointing device, a mouse, a trackball, a joystick, a camera, amicrophone, earphones or buds with input functionality, a gamingcontroller, or other connectable input device. A user interacting withan input device included on the computing device 750 when the computingdevice is incorporated into the VR space can cause a particular actionto occur in the VR space.

In some implementations, a touchscreen of the computing device 750 canbe rendered as a touchpad in VR space. A user can interact with thetouchscreen of the computing device 750. The interactions are rendered,in VR headset 785 for example, as movements on the rendered touchpad inthe VR space. The rendered movements can control objects in the VRspace.

In some implementations, one or more output devices included on thecomputing device 750 can provide output and/or feedback to a user of theVR headset 785 in the VR space. The output and feedback can be visual,tactical, or audio. The output and/or feedback can include, but is notlimited to, vibrations, turning on and off or blinking and/or flashingof one or more lights or strobes, sounding an alarm, playing a chime,playing a song, and playing of an audio file. The output devices caninclude, but are not limited to, vibration motors, vibration coils,piezoelectric devices, electrostatic devices, light emitting diodes(LEDs), strobes, and speakers.

In some implementations, the computing device 750 may appear as anotherobject in a computer-generated, 3D environment. Interactions by the userwith the computing device 750 (e.g., rotating, shaking, touching atouchscreen, swiping a finger across a touch screen) can be interpretedas interactions with the object in the VR space. In the example of thelaser pointer in a VR space, the computing device 750 appears as avirtual laser pointer in the computer-generated, 3D environment. As theuser manipulates the computing device 750, the user in the VR space seesmovement of the laser pointer. The user receives feedback frominteractions with the computing device 750 in the VR space on thecomputing device 750 or on the VR headset 785.

A number of embodiments have been described. Nevertheless, it will beunderstood that various modifications may be made without departing fromthe spirit and scope of the invention.

In addition, the logic flows depicted in the figures do not require theparticular order shown, or sequential order, to achieve desirableresults. In addition, other steps may be provided, or steps may beeliminated, from the described flows, and other components may be addedto, or removed from, the described systems. Accordingly, otherembodiments are within the scope of the following claims.

What is claimed is:
 1. A method comprising: presenting an augmentedreality environment on a display of a first device, the augmentedreality environment including at least an image captured using an imagesensor of the first device and at least a first virtual object appliedto the image; receiving, in the first device, a relative descriptionfile for physical markers that are positioned at locations, the relativedescription file defining relative positions for each of the physicalmarkers with regard to at least another one of the physical markers;initially localizing a position of the first device among the physicalmarkers by aiming the image sensor in a first direction toward a firstphysical marker and visually capturing the first physical marker of thephysical markers using the image sensor of the first device; recognizinga second physical marker of the physical markers and a location of thesecond physical marker without a line of sight between the image sensorand the second physical marker, the second physical marker recognizedusing the relative description file, wherein a user moves the firstdevice to the location of the second physical marker, and wherein at thelocation of the second physical marker the first physical marker is nolonger visible in the augmented reality environment and the user aimsthe image sensor in a second direction toward the location of the firstphysical marker without a line of sight between the image sensor and thefirst physical marker; and updating the augmented reality environmentwith the first virtual object at a predefined location relative to thefirst physical marker.
 2. The method of claim 1, wherein at least thefirst physical marker comprises a code.
 3. The method of claim 1,wherein at least the first physical marker comprises a natural marker.4. The method of claim 1, wherein the first virtual object is positionedin the image at a predefined location relative to the first physicalmarker.
 5. The method of claim 1, wherein, after the position of thefirst device is initially localized, the user aims the image sensor in athird direction, and wherein recognizing the second physical markercomprises determining, using the relative description file, that thelocation of the second physical marker is located in the thirddirection.
 6. The method of claim 5, further comprising updating theaugmented reality environment with a second virtual object in apredefined position relative to the second physical marker.
 7. Themethod of claim 5, wherein before the second physical marker isrecognized, the second physical marker is removed from the location ofthe second physical marker, and wherein the second physical marker andthe location of the second physical marker are recognized using therelative description file.
 8. The method of claim 1, wherein initiallylocalizing the position of the first device comprises centering acoordinate system of the first device based on the location of the firstphysical marker, and wherein recognizing the second physical marker doesnot comprise re-centering the coordinate system.
 9. The method of claim8, wherein centering the coordinate system of the first device comprisesrecalibrating a center of the coordinate system to correct a trackingerror.
 10. The method of claim 1, wherein initially localizing theposition of the first device comprises centering a coordinate system ofthe first device based on the location of the first physical marker,wherein recognizing the second physical marker comprises re-centeringthe coordinate system, and wherein any virtual objects continue to bepresented in a same space via camera passthrough because relativecoordinates are proportionally the same.
 11. The method of claim 1,further comprising providing the relative description file to a seconddevice, wherein the second device is configured to initially localize aposition of the second device among the physical markers by visuallycapturing any one of the physical markers using an image sensor of thesecond device, and to use the relative description file to recognizelocations of all remaining ones of the physical markers without a lineof sight.
 12. The method of claim 1, wherein the relative descriptionfile is a first relative description file, the method further comprisingreceiving a second relative description file from a second device, andat least partially merging the first and second relative descriptionfiles.
 13. The method of claim 1, wherein the relative description filedefines the relative position of the second physical marker with regardto the first physical marker, the relative position comprising adisplacement of the second physical marker relative to the firstphysical marker.
 14. The method of claim 1, wherein the first devicerecognizes at least some of the physical markers, the method furthercomprising generating a log on the first device reflecting the at leastsome of the physical markers that the first device has recognized. 15.The method of claim 1, wherein the relative description file defines therelative position of each of the physical markers with regard to allother ones of the physical markers.
 16. A non-transitory storage mediumhaving stored thereon instructions that when executed are configured tocause a processor to perform operations, the operations comprising:presenting an augmented reality environment on a display of a firstdevice, the augmented reality environment including at least an imagecaptured using an image sensor of the first device and at least a firstvirtual object applied to the image; receiving, in the first device, arelative description file for physical markers that are positioned atlocations, the relative description file defining relative positions foreach of the physical markers with regard to at least another one of thephysical markers; initially localizing a position of the first deviceamong the physical markers by aiming the image sensor in a firstdirection toward a first physical marker and visually capturing thefirst physical marker of the physical markers using the image sensor ofthe first device; recognizing a second physical marker of the physicalmarkers and a location of the second physical marker without a line ofsight between the image sensor and the second physical marker, thesecond physical marker recognized using the relative description file,wherein a user moves the first device to the location of the secondphysical marker, and wherein at the location of the second physicalmarker the first physical marker is no longer visible in the augmentedreality environment and the user aims the image sensor in a seconddirection toward the location of the first physical marker without aline of sight between the image sensor and the first physical marker;and updating the augmented reality environment with the first virtualobject at a predefined location relative to the first physical marker.17. A device comprising: at least one processor; and a non-transitorystorage medium having stored thereon: a first relative description filefor physical markers that are positioned at locations, the firstrelative description file defining relative positions for each of thephysical markers with regard to at least another one of the physicalmarkers; and instructions that when executed are configured to cause theprocessor to perform operations, the operations comprising: presentingan augmented reality environment on a display of a device, the augmentedreality environment including at least an image captured using an imagesensor of the device and at least a first virtual object applied to theimage; initially localizing a position of the device among the physicalmarkers by aiming the image sensor in a first direction toward a firstphysical marker and visually capturing the first physical marker of thephysical markers using the image sensor of the device; recognizing asecond physical marker of the physical markers and a location of thesecond physical marker without a line of sight between the image sensorand the second physical marker, the second physical marker recognizedusing the first relative description file, wherein a user moves thedevice to the location of the second physical marker, and wherein at thelocation of the second physical marker the first physical marker is nolonger visible in the augmented reality environment and the user aimsthe image sensor in a second direction toward the location of the firstphysical marker without a line of sight between the image sensor and thefirst physical marker; and updating the augmented reality environmentwith the first virtual object at a predefined location relative to thefirst physical marker.
 18. The device of claim 17, wherein the firstrelative description file defines a first relative position thatconnects the first and second physical markers, and wherein the firstrelative description file does not define a second relative positionthat connects the second physical marker and a third physical marker.19. The device of claim 18, wherein the operations further comprise:receiving a second relative description file that defines the secondrelative position, and at least in part merging the first and secondrelative description files to a third relative description file thatreflects the first and second relative positions.